Skip to content

fix(models): use standard tool role for Gemma 4 on Vertex AI - #7978

Closed
rladeira wants to merge 1 commit into
google:mainfrom
rladeira:fix/gemma4-vertex-maas-tool-role
Closed

rladeira wants to merge 1 commit into
google:mainfrom
rladeira:fix/gemma4-vertex-maas-tool-role

Conversation

@rladeira

@rladeira rladeira commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

2. Or, if no issue exists, describe the change:

Problem:
_is_gemma4_model in google/adk/models/lite_llm.py causes ADK to send tool results with role="tool_responses" whenever the model name matches gemma-?4, unless the effective provider is in _STANDARD_TOOL_ROLE_PROVIDERS ({"openai", "azure", "lm_studio"}, added in 7d190e6e for #6482).

When calling Gemma 4 on Vertex AI Model Garden MaaS via LiteLLM (vertex_ai/google/gemma-4-26b-a4b-it-maas), LiteLLM dispatches to Vertex AI's OpenAI-compatible chat-completions endpoint (.../endpoints/openapi/chat/completions), which strictly validates OpenAI message roles and rejects role="tool_responses" with 400 INVALID_ARGUMENT ("Unrecognized 'role' field for a message; found: 'tool_responses'.").

Solution:
Add "vertex_ai" to _STANDARD_TOOL_ROLE_PROVIDERS and update the _is_gemma4_model docstring and _content_to_message_param comment:

  • Follows the provider-set exclusion pattern established in 7d190e6e with no public API changes.
  • Fixes tool-result turns on Vertex AI Model Garden MaaS (vertex_ai/google/gemma-4-*) and custom_llm_provider="vertex_ai" routes, including _ensure_tool_results healing.
  • Verified against both Vertex AI Model Garden MaaS and a self-deployed Vertex AI Model Garden Gemma 4 vLLM endpoint (google/gemma4@gemma-4-e2b-it via vertex_ai/gemma/... :predict with @requestFormat="chatCompletions" and dedicated /chat/completions), which also accepts role="tool" cleanly.
  • Preserves role="tool_responses" for ollama/... and hosted_vllm/....

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

Added parametrized cases in tests/unittests/models/test_lite_llm_gemma_tool_role.py covering vertex_ai/google/gemma-4-26b-a4b-it-maas and custom_llm_provider="vertex_ai" across TestLiteLlmGenerateContent and TestEnsureToolResults.

pytest summary:

$ uv run pytest tests/unittests/models/test_lite_llm_gemma_tool_role.py
46 passed in 13.93s

$ uv run pytest tests/unittests/models/
1884 passed, 31 warnings in 50.60s

$ uv run pytest tests/unittests/test_import_loading.py
29 passed in 66.42s

Manual End-to-End (E2E) Tests:

  1. Live tool-calling round trip against Vertex AI Model Garden MaaS (vertex_ai/google/gemma-4-26b-a4b-it-maas, vertex_location="global"):
# Before (main):
roles sent: ['user', 'assistant', 'tool_responses']
EXCEPTION: BadRequestError litellm.BadRequestError: Vertex_aiException BadRequestError - [{
  "error": {
    "code": 400,
    "message": "Unrecognized 'role' field for a message; found: 'tool_responses'.",
    "status": "INVALID_ARGUMENT"
  }
}]

# After (this PR):
roles sent: ['user', 'assistant', 'tool']
error_code: None | error_message: None
finish_reason: FinishReason.STOP | text: 'The weather in Paris is currently sunny with a temperature of 21°C.'
  1. Live 2-turn tool-calling round trip against a self-deployed Vertex AI Model Garden Gemma 4 vLLM endpoint (google/gemma4@gemma-4-e2b-it via vertex_ai/gemma/gemma-4-e2b-it :predict and dedicated /chat/completions):
TURN 1 emitted function_call: id='chatcmpl-tool-8ea155d37d285a94' name='get_weather' args={'city': 'Paris'}
TURN 2 OK | roles_sent=['user', 'assistant', 'tool'] | finish=FinishReason.STOP | tool_calls=[] | text='The weather in Paris is sunny with a temperature of 21 C.'

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

Additional context

None.

Vertex AI Model Garden MaaS serves Gemma 4 via Vertex AI's
OpenAI-compatible chat-completions endpoint, which rejects
role='tool_responses' with 400 INVALID_ARGUMENT ("Unrecognized 'role'
field for a message; found: 'tool_responses'."). Self-deployed Vertex AI
Model Garden Gemma 4 endpoints also accept role='tool'.

Add 'vertex_ai' to _STANDARD_TOOL_ROLE_PROVIDERS so Gemma 4 on Vertex AI
sends standard role='tool' while preserving 'tool_responses' for Ollama
and hosted_vllm.

Fixes google#7977
@rladeira
rladeira force-pushed the fix/gemma4-vertex-maas-tool-role branch from 5c8a013 to d5cace7 Compare November 3, 2026 20:46
copybara-service Bot pushed a commit that referenced this pull request Nov 5, 2026
Merge #7978

LiteLlm now sends tool results for Gemma 4 models on the vertex_ai provider
(model prefix or custom_llm_provider) with role "tool" instead of
"tool_responses", which Vertex AI's OpenAI-compatible endpoint rejects.

Fixes #7977

PiperOrigin-RevId: 1014964659
@adk-bot

adk-bot commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

Thank you @rladeira for your contribution! 🎉

Your changes have been successfully imported and merged via Copybara in commit b33b43e.

Closing this PR as the changes are now in the main branch.

@adk-bot adk-bot added the merged [Status] This PR is merged label Nov 5, 2026
@adk-bot adk-bot closed this Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merged [Status] This PR is merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[LiteLLM] role='tool_responses' for Gemma 4 breaks tool calling on Vertex AI Model Garden MaaS (400 INVALID_ARGUMENT)

3 participants